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Introduction 


In  the  course  of  this  research  project  we  have  been  exploring  the  integration  of  Immersive  and 
Non-Immersive  systems.  This  development  and  research  has  lead  to  a  number  of  innovative 
ideas  and  system  configurations.  In  particular,  we  have  developed  prototypes  which  make 
models  shown  on  the  Non-Immersive  system  appear  more  real;  pushed  the  development  of 
alternative  input  devices  for  interaction  with  three  dimensional  models;  and  gained  more  insight 
into  the  nature  of  presence.  In  addition  to  prototype  hardware  and  software  systems,  our  research 
has  also  lead  to  a  potentially  new  method  for  supporting  several  users  around  a  Non-Immersive 
virtual  workbench  type  display.  The  capability  to  support  several  users  clustered  around  the 
Non-Immersive  system  will  permit  very  powerful  future  applications.  One  can  envisage  several 
users  around  the  Non-Immersive  system  evaluating  a  particular  situation.  They  can  then  dispatch 
particular  areas  or  features  for  evaluation  in  detail  by  other  users  who  are  looking  at  the  same 
database  with  the  Immersive  workstation.  This  type  of  hybrid  cooperative  application  is  very 
promising  and  could  be  performed  at  a  single  site  or  over  a  network  consisting  of  several  sites. 

The  basic  system  configuration  for  the  Immersive  Workstation  includes  a  high  resolution 
stereoscopic  display  on  a  compliant  base.  The  compliant  base  implements  a  unique  navigational 
paradigm  which  uses  the  body’s  axial  muscles  to  facilitate  navigation  in  the  virtual  space.  This 
navigational  interface  permits  the  user  to  control  their  position  with  ease,  at  both  a  macro  and  a 
micro  scale  without  changing  modes.  The  Immersive  Workstation  provides  the  user  with  an 
immersive  experience  and  does  not  require  any  head  gear  to  be  worn.  One  can  easily  sit  down 
and  use  the  station  without  having  to  make  any  significant  adjustments. 


User  at  the  immersive  Workstation 

The  Non-Immersive  Workstation  user  also  sees  high  resolution  stereoscopic  images  but  in  a  very 
different  way.  The  Non-immersive  system  is  based  on  a  large  rear  projection  table  type  display. 
The  user  views  the  stereo  images  on  the  display  surface  and  navigates  using  the  same 
navigational  paradigm  as  the  Immersive  user.  The  actual  input  device  which  implements  the 
interface  is  different.  Several  people  could  stand  around  the  display  to  point  and  discuss  the 
virtual  model  projected  onto  the  table  surface.  Using  currently  available  technology,  two 
interactive  correct  stereo  perspective  views  can  be  supported.  In  the  course  of  our  research,  we 
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have  found  a  method  for  extending  this  to  a  larger  number  of  simultaneous  viewers.  This  future 
system  would  also  have  the  potential  to  be  self  contained,  rugged,  and  deployable  in  a  shipboard 
environment. 


User  at  the  Non-Immersive  Workstation 


Both  the  Non-Immersive  and  Immersive  workstations  are  driven  by  a  single  Silicon  Graphics 
Machine  (Crimson  RE  with  Multi-Channel  Option).  The  Multi-Channel  Option  (MCO) 
simultaneously  drives  both  the  Non-Immersive  and  the  Immersive  displays  in  stereo.  All  the 
trackers  and  other  input  devices  are  connected  to  the  serial  ports  of  the  machine.  The  prototype 
software  developed  and  used  in  the  research  we  performed  is  written  in  C  using  the  SGI 
Performer  Libraries.  While  the  current  software  implementation  is  merely  a  prototype,  it  would 
be  possible  to  extend  the  software  to  permit  the  various  users  of  the  Immersive  and  Non- 
Immersive  workstations  to  be  on  different  machines.  By  using  a  standard  networked 
architecture,  the  graphics  performance  of  the  various  stations  could  be  improved  because  each 
user  could  have  a  dedicated  machine,  rather  than  relying  on  a  central  resource. 


Key  Developments 


Powerful  Simultaneous  Multi-User  Capability 

The  Non-Immersive  display  can  be  expected  to  be  used  by  several  people  simultaneously.  This 
therefore  requires  that  several  head-tracked  stereoscopic  views  be  presented  on  the  display 
surface  simultaneously.  Fakespace  has  developed  a  system  for  two  simultaneous  users  based  on 
CRT  projectors.  Building  on  that  experience,  we  have  developed  an  approach  for  using  an 
alternative  projection  technology  which,  it  is  expected,  could  support  several  simultaneous 
viewers  at  once.  This  is  very  exciting  because  the  ability  for  people  to  share  information  is 
critical. 


Four  Users  Looking  at  a  cube 
(note  their  radically  different  perspectives) 


This  new  approach  would  also  permit  the  display  of  different  information  to  selected  users. 

There  are  many  possible  scenarios  which  take  advantage  of  this  unique  capability.  For  example: 

Several  Users  -  Overlays:  Several  people  are  discussing  a  particular  area  of  a  map  and 
the  various  features  of  the  terrain.  One  user  might  want  to  call  up  a  particular  satellite 
image  of  the  area.  The  overlaid  image  can  be  made  to  appear  in  only  that  person’s  view. 
This  is  beneficial  because  while  all  users  share  a  common  underlying  database,  each  user 
can  overlay  information  specific  to  their  interests  and  expertise.  Overlays  showing 
suspected  mine  fields  might  be  very  relevant  to  someone  commanding  ground  forces 
while  less  important  to  the  person  coordinating  air  support. 

User  Specific  Queries:  One  user  may  wish  to  see  information  pertaining  to  a  particular 
entity.  They  may  pick  the  entity  (either  using  a  virtual  laser  pointer,  or  by  pointing  into 
the  model  with  their  hand).  The  status  window  then  appears.  That  window  need  not 
impact  anybody  else’s  view  of  the  data  base. 

While  this  new  multi-person  perspective  system  has  not  been  proven  yet,  we  are  confident  that 
more  than  two  simultaneous  users  could  be  supported. 

This  new  system  would  incorporate  projection  technologies  which  produce  stereo  images  from 
one  or  two  lenses.  The  projectors  are  also  physically  smaller  therefore  lending  themselves  to 
being  located  underneath  the  table.  Ideally,  the  projectors  would  be  located  entirely  beneath  the 
table  so  that  users  may  walk  around  the  display  and  view  it  from  any  perspective. 


Prototype  Multi-User  Immersive  /  Non-lmmersive  Software 

The  prototype  softw'are  written  as  part  of  this  research  effort  permits  two  users  to  explore  the 
same  database  in  two  very  different  ways.  The  user  at  the  Non-lmmersive  workstation  sees  the 
database  from  a  “God’s  eye”  perspective.  The  Immersive  user’s  view  is  immersive  in  that  they 
are  in  the  scene.  An  icon  can  be  used  to  show  the  position  and  orientation  of  the  Immersive 
viewer  to  the  Non-lmmersive  observer. 


Non-lmmersive  View  Immersive  View 
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Longer  Range  Acoustic  Tracking 


Magnetic  tracking  systems  are  not  well  suited  to  ship  board  use  because  of  the  metallic  nature  of 
the  environment.  One  alternative  technology  is  ultrasonic  tracking.  In  trying  to  use  the  Logitech 
3D  Mouse  systems  for  head-tracking,  the  range  was  found  to  be  inadequate  so  the  firmware  of 
the  tracker  was  altered  to  increase  the  range.  In  addition  to  nice  smooth  tracking,  the  receiver 
can  be  integrated  into  the  standard  Stereo  Graphics  glasses  as  shown  below. 


Tracker  in  the  frame  of  the  glasses 
(The  three  small  black  disks  are  the  tracker’s  receiver) 


While  acoustic  tracking  is  probably  not  ideal  for  a  naval  environment,  it  could  be  used  in 
conjunction  with  magnetic  tracking  to  obtain  the  desired  tracking  accuracy,  smoothness,  and 
precision.  In  addition,  we  were  pleasantly  surprised  that  the  line  of  sight  nature  of  the  acoustic 
tracking  was  not  objectionable.  This  make  us  confident  that  an  optical  IR  line  of  sight  tracker 
would  also  work  reasonably  well  and  would  be  less  sensitive  to  external  interference  than  either 
the  acoustic  or  magnetic  tracking  alternatives.  Both  the  acoustic  and  potentially  the  IR  methods 
can  be  made  wireless  which  would  be  a  great  improvement. 
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New  Interaction  Devices 


In  developing  the  interaction  paradigms  for  the  Immersive  and  Non-Immersive  workstation,  we 
concentrated  on  the  Non-Immersive  user.  In  order  to  provide  the  Non-Immersive  user  with 
selection  and  interaction  tools,  we  developed  the  idea  of  clear  touch  tablets  (called  V-palettes) 
which  are  held  by  the  user.  These  tablets  provide  the  user  with  menus  which  float  in  space  and 
are  activated  by  touching  the  appropriate  area  on  the  V-palette. 


Renderings  of  several  V-palettes 
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New  Configurations  for  Non-lmmersive  Displays 

In  trying  to  achieve  the  most  spatial  images  on  the  Non-lmmersive  display,  we  found  that  one 
technique  for  creating  more  of  an  impression  of  space  was  to  raise  the  perimeter  of  the  display 
area  so  that  one  can  rest  ones  hands  on  the  edge  and  reach  into  the  three  dimensional  model 
space. 
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Consistent  Navigational  Paradigms 


In  developing  a  prototype  which  provides  a  consistent  user  interface  to  the  Immersive  and  Non- 
Immersive  workstations,  the  axial  muscle  based  Push  interface  was  used.  This  interface  provides 
a  high  degree  of  control  over  many  orders  of  magnitude  without  mode  changes.  It  also  provides 
capabilities  from  a  navigational  perspective  which  are  particularly  beneficial  to  the  Immersive 
user.  The  desktop  Push  frees  the  immersive  user  to  look  in  any  direction  while  moving.  For 
example,  it  is  easy  to  move  down  a  street  while  turning  to  look  at  the  buildings  to  the  right  or  left 
without  deviation  from  one’s  desired  motion  down  the  street. 

The  PushStick  is  used  by  the  Non-Immersive  user.  The  PushStick  implements  the  same  type  of 
navigational  interface  as  the  desktop  Push  display  and  is  based  on  non-linear  axial  muscle 
control.  The  use  of  the  same  fundamental  type  of  navigational  paradigm  leads  to  a  consistent 
feel  when  moving  from  one  workstation  to  the  other. 


PushStick  Desktop  Push 
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Conclusion 


In  conclusion,  much  progress  has  been  made  towards  the  integration  of  the  Non-Immersive  and 
Immersive  Workstations.  In  particular,  we  have  prototyped  various  software  and  hardware 
components  to  accomplish  this. 

In  our  research  on  ship-compatible  tracking  we  experimented  with  acoustic  tracking  for  the  Non- 
Immersive  user  and  found  that  line  of  sight  tracking  is  less  problematic  than  one  might  expect  on 
the  basis  of  people’s  experiences  with  head  mounted  displays.  The  Immersive  workstation 
tracking  is  mechanical  and  thus  not  subject  to  any  interference  from  normal  external  sources. 

The  research  into  the  presentation  of  imagery  on  the  Non-Immersive  display  leads  us  to  believe 
that  for  a  single  viewer,  passive  glasses  could  be  used  in  conjunction  with  optical  tracking  thus 
totally  freeing  the  user  from  wire  and  batteries.  For  the  multi-person  stereo  system  we  expect 
that  the  glasses  and  tracking  can  be  made  wireless  but  not  passive. 

The  Immersive  user  workstation  is  compact  and  suitable  for  seated  use  which  addresses  the 
safety  issues  one  might  come  up  against  with  traditional  head  mounted  displays  or  Boom  type 
displays  used  on  board  a  ship. 

The  foundation  developed  for  the  future  realization  of  a  Non-Immersive  head-tracked  stereo 
system  capable  of  supporting  a  number  of  simultaneous  viewers  is  very  promising.  We  believe 
such  a  system  would  potentially  have  a  much  smaller  physical  footprint  than  the  current  Non- 
Immersive  system. 

We  look  forward  to  continuing  this  effort  and  developing  the  multi-person  capability  into  a  tool 
which  we  believe  will  fundamentally  change  the  way  command  and  control  systems  function  in 
the  future.  There  is  a  wealth  of  computer  gathered  data.  The  timely  and  intelligent  presentation 
of  this  data  for  decision  makers  is  crucial  for  making  the  best  possible  decisions  with  confidence 
and  speed. 
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